Li  12.8 

Uj 

lit 
IU 
ui 


1.25  ■  1.4 


1.0  ta 


MICROCOPY  RESOLUTION  TEST  CHART 
NATIONAL  BUREAU  OF  STANDAR0S-1963-A 


MICROCOPY  RESOLUTION  TEST  CHART 
NATIONAL  BUREAU  OF  STANDARDS- 1963- A 


114  L £ 


|£8  |25 

1. 

l££  |Z2 

M3A  ■■■ 

i . 

Li  ^0 

C..1 

MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  OF  STANDARDS -1963- A 


u  Ka 

Li  «■"“ 


1.0  S“ 

m  13^2 


MICROCOPY  RESOLUTION  TEST  CHART 
NATIONAL  BUREAU  OF  STANOAROS-1963-A 


MICROCOPY  RESOLUTION  TEST  CHART 
NATIONAL  BUREAU  Of  STAN0AR00-1963-A 


ADA120722 


•  1 


\ 

o 

00 

r* 


o 
(/) 
%  o 


< 


Technical  Report  780 

CONSISTENT  AUTOREGRESSIVE  SPECTRAL 
ESTIMATION  FOR  NOISE-CORRUPTED 
AUTOREGRESSIVE  TIME  SERIES 

D.F.  Gingras 

30  March  1982 

Final  Report:  October  1981-March  1982 

Prepared  for 
Naval  Material  Command 


o 

to 

o 

— i 
30 

->4 

00 

O 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


Ui^i  i  .?  ij62 

A 


NAVAL  OCEAN  SYSTEMS  CENTER 

San  Diego,  California  92152 


NAVAL  OCEAN  SYSTEMS  CENTER.  SAN  DIEGO.  CA  921S2 


AN  ACTIVITY  OF  THE  NAVAL  MATERIAL  COMMAND 
SL  GUILLE,  CAPT,  USN  HL  BLOOD 


Commandar 


HL  BLOOD 

Tachnical  Diractor 


ADMINISTRATIVE  INFORMATION 


This  work  was  performed  from  October  1981  through  March  1982  and  was 
sponsored  by  NAVMAT.  Funding  was  provided  through  Program  Element  61 152N,  Project 
ZR00001,  Task  Area  ZRO 1408 13,  and  Work  Unit  713-ZS54. 


Released  by 
R.  R.  Smith,  Head 
Signal  Processing  and 
Display  Division 


Under  authority  of 
H.  A.  Schenck,  Head 
Undersea  Surveillance 
Department 


SECURITY  CLASSIFICATION  OF  THIS  PACE  (Whan  Data  Entered) 


r.  REPORT  number 


REPORT  DOCUMENTATION  PAGE  1  beforePcompletiwg^form 


2.  GOVT  ACCESSION  NO.  S.  RECIPIENT'S  CATALOG  NUMBER 


NOSC  Technical  Report  780  (TR  780)  l  A!&£> 


4.  TITLE  (and  Submit) 

Consistent  Autoregressive  Spectral  Estimation  for  Noise-Corrupted 
Autoregressive  Time  Series 


S  TYPE  OF  REPORT  A  PERIOD  COVERED 

Final  Report:  October  1981  - 
March  1982 


C.  PERFORMING  0R6.  REPORT  NUMBER 


7.  AU  THORf  a) 


D.  F.  Gingras 


S.  PERFORMING  ORGANIZATION  NAME  AND  AOORESS 

Naval  Ocean  Systems  Center 
San  Diego,  CA  921S2 


II.  CONTROLLING  OFFICE  NAME  AND  AOORESS 

Naval  Material  Command 
Washington,  D.C. 


I.  CONTRACT  OR  GRANT  NUMBERS) 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  A  WORK  UNIT  NUMBERS 

61 152N,  ZR00001 ,  ZR0140813 
713  -  ZSS4 


12.  REPORT  DATE 

30  March  1982 


<}.  NUMBER  OF  PAGES 
21 


4.  MONITORING  AGENCY  NAME  A  ADDRESS^!/  dlttarant  from  Controlling  Ofllca)  IS.  SECURITY  CLASS,  (at  thta  raport) 

Unclassified 


IS*.  DECLASSIFICATION/  DOWNGRADING 
SCHEDULE 


16.  DISTRIBUTION  STATEMENT  (ol  thla  Raport) 

Approved  for  public  release;  distribution  unlimited. 


17.  DISTRIBUTION  STATEMENT  (ol  tha  abatraet  entered  In  Block  20,  II  d!  Iterant  Iron  Raport) 


If.  KEY  WORDS  ( Comtlnua  on  ravaraa  alda  II  nacaaaary  and  Idantlly  by  bloc*  number) 


spectral  estimation 
autoregressive  processes 


noise  corruption 
time  series  analysis 


ABSTRACT  fConlfnu*  on  ravaraa  alda  II  nacaaaary  and  Idantlly  by  block  numbat) 

For  the  case  when  the  observed  series  consists  of  the  sum  of  an  autoregressive  process  of  known  order  and  white 
noise  the  application  of  autoregressive  spectral  estimation  methods  may  not  be  correct.  The  presence  of  the 
additive  noise  introduces  zeros  which  are  not  adequately  modeled  by  an  autoregressive  model.  In  this  report  an 
autoregressive  spectral  estimator  for  the  noise-corrupted  case  is  developed  and  shown  to  be  consistent.  The 
-'’high-order^  Yule-Walker  equations  are  used  to  estimate  the  autoregressive  parameters  from  the  noise-corrupted 
observations.  A  least  squares  estimate  for  the  variance  of  the  innovations  sequence  is  also  developed  and  shown 
to  be  consistent.  These  consistent  estimates  for  the  autoregressive  parameters  and  the  innovations  variance  are 
used  to  form  the  consistent  autoregressive  spectral  estimates. 


FORM 
I  JAN  71 


EDITION  OF  1  NOV  AS  IS  OBSOLETE 

$/N  0102-  LF-0I4-660I 


UNCLASSIFIED _ 

SECURITY  CLASSIFICATION  OF  THIS  PAOE 


(Whan  Data  Entered) 


SCCUftlTY  CLAUDICATION  OF  THIS  FAOEflThMi  D< 


iH 


SUMMARY 


It  has  been  shown  that  the  application  of  AR  spectral  estimation  methods  to  time 
series  consisting  of  an  AR  process  observed  in  white  noise  is  not  a  correct  application  for  the 
AR  method.  An  autoregressive  process  plus  noise  (AR  +  N)  has  been  shown  to  require  an 
ARMA  model  for  correct  representation.  The  “higher-order”  Yule-Walker  equations  were 
shown  to  provide  consistent  and  unbiased  estimates  for  the  AR  parameters  of  an  AR  +  N 
series.  A  consistent  estimate  for  the  innovations  variance  of  the  unobservable  AR  process  was 
also  developed.  The  resulting  spectral  estimate  for  the  AR  series  as  observed  in  white  noise 
was  shown  to  be  a  consistent  estimate.  It  now  remains  to  be  shown  that  the  AR  spectral 
estimate  is  unbiased  and  asymptotically  normal.  The  structure  of  the  asymptotic  covariance 
matrix  must  also  be  evaluated. 
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I.  INTRODUCTION 
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Parzen  [  1  ]  and  Burg  (2] ,  [3]  proposed  high  resolution  methods  of  power  spectral 
estimation.  Parzen’s  method  is  referred  to  as  the  autoregressive  (AR)  method  and  Burg’s 
method  as  the  maximum  entropy  method  (MEM).  The  principal  advantage  of  the  AR  and 
MEM  methods  over  conventional  smoothed  periodogram  methods  is  their  enhanced  resolu¬ 
tion  property.  A  secondary  advantage,  related  to  the  enhanced  resolution  property,  is  that 
these  methods  do  not  require  the  use  of  a  window  function  to  achieve  a  consistent  estimate. 
This  results  in  both  the  resolution  and  stability  being  more  directly  related  to  the  character¬ 
istics  of  the  time  series  under  observation.  Van  den  Bos  [4]  showed  that  MEM  spectral 
analysis  is  equivalent  to  fitting  an  AR  model  to  the  series  thus  establishing  an  equivalence 
between  the  two  methods.  Only  the  AR  method  will  be  discussed  in  this  report. 

Often  the  observed  time  series,  Yt,  is  not  generated  by  a  pure  AR  process  but  instead 
by  the  sum  of  an  AR  process  and  white  noise.  Let  Xt  be  an  AR  process  and,  nt,  white  noise, 
then  the  observed  time  series  is  given  by 

Yt  -  Xt  +  nt 

and  will  be  referred  to  as  an  AR  +  N  process.  The  application  of  AR  spectral  estimation 
methods  assumes  that  the  observed  time  series  can  be  fitted  to  a  finite  order  AR  process, 
that  is,  an  all-pole  spectral  model.  Pagano  15]  showed  that  the  correct  model  for  an  AR 
process  plus  noise  is  an  autoregressive-moving  average  ARMA,  pole-zero  spectral  model. 
Thus,  when  additive  noise  is  present,  the  all-pole  model  assumption  is  violated,  resulting 
in  a  degraded  spectral  estimate,  Kay  [6] .  It  has  been  shown  experimentally  that  much  of 
the  enhanced  resolution  property  is  not  realized  when  AR  spectral  estimation  methods  are 
applied  to  an  AR  process  in  noise,  Marple  [7] .  The  problem  under  investigation  herein  is 
the  development  of  a  consistent  autoregressive  spectral  estimator  that  takes  into  account 
the  presence  of  additive  noise. 


1  Parzen,  E.,  “Statistical  spectrum  analysis  (single  channel  case)  in  1968.”  Stanford  Univ.  Tech.  Report 
#  1 1  on  Contract  Nonr-225(80),  1968. 

^  Burg,  J.P.,  “Maximum  entropy  spectral  analysis,”  37th  Annual  Int.  Meeting  Soc.  of  the  Explor. 
Geophys.,  Oklahoma  City,  1967. 

^  Burg,  J.P.,  “A  new  analysis  technique  for  time  series  data,”  Adv.  Study  Institute  on  Signal  Processing, 
NATO,  Enschede,  Netherlands,  1968. 

4  van  den  Bos,  A.,  “Alternative  interpretation  of  maximum  entropy  spectral  analysis,”  IEEE  Trans. 

Inform.  Theory,  IT- 17, 493-494,  1971. 

^  Pagano,  M.,  “Estimation  of  models  of  autoregressive  signal  plus  white  noise.”  Ann.  Statist.,  vol  12, 
no.  1,99-108, 1974. 

6  Kay,  S.M.,  “The  effects  of  noise  on  the  autoregressive  spectral  estimator,  IEEE  Trans.  Acoust.,  Speech, 
Signal  Processing,  vol  ASSP-27,  no.  5, 478-485, 1979. 

7 

Marple,  S.L.,  “Resolution  of  conventional  Fourier,  autoregressive  and  special  ARMA  methods  of 
spectral  analysis.”  Presented  at  IEEE  Int.  Conf.  Acoustics,  Speech  and  Signal  Proc.,  Hartford,  CT,  1977. 
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For  autoregressive  spectral  estimation,  without  additive  noise,  methods  exist  for 
estimating  the  AR  parameters  and  thus  the  spectrum.  Asymptotic  First*  and  second-order 
statistics,  under  varying  conditions  on  the  underlying  process,  have  been  derived  by  Akaike 
[8] ,  Kromer  [9] ,  and  Berk  [  10] .  Overall  the  results  indicate  that,  under  proper  conditions, 
the  AR  spectral  estimates  have  asymptotic  statistical  properties  similar  to  a  smoothed 
periodogram  with  a  truncated  rectangular  window  function.  Pagano  [5]  investigated  the 
estimation  of  AR  parameters  in  noise  and  showed  that  strongly  consistent  efficient  estimates 
for  the  AR  parameters  could  be  obtained,  but  he  did  not  address  the  problem  of  statistical 
properties  for  the  spectral  estimate.  Kay  [11]  discusses  a  method  that  compensates  for  the 
presence  of  noise  in  AR  spectral  estimation,  but  his  method  requires  knowledge  of  the  noise 
variance.  Also,  he  does  not  develop  statistical  properties  for  the  AR  parameter  estimates  or 
for  the  AR  spectral  estimates  for  his  noise-compensated  estimates. 

In  this  report  it  is  shown  that  the  correct  model  for  an  AR  +  N  series  is  a  special 
form  of  the  general  ARMA  model.  In  a  previous  report,  Gingras  [  1 2] ,  it  was  shown  that 
when  the  “higher  order”  Yule-Walker  (Y-W)  equations  are  used  to  estimate  the  AR  para¬ 
meters  for  an  AR  +  N  series,  the  estimates  are  consistent  and  asymptotically  normal.  In 
this  report  it  is  shown  that  those  consistent  AR  parameter  estimates,  along  with  a  consistent 
least  squares  estimate  for  the  variance  of  the  AR  innovations  series  form  the  basis  for  a  con¬ 
sistent  estimate  of  the  AR  process  spectrum.  The  asymptotic  statistics  for  the  proposed 
spectral  estimate  are  under  investigation  and  will  be  presented  in  a  future  report. 

In  Section  II  the  AR  and  ARMA  processes  and  their  power  spectral  densities  are  pre¬ 
sented.  Estimates  for  the  AR  parameters  are  discussed  for  both  cases.  It  is  shown,  in  Section 
III,  that  an  autoregressive  process  plus  white  noise  series,  AR  +  N,  is  a  special  case  of  the 
more  general  ARMA  series  and  that  the  “higher  order”  Y-W  equations  can  be  used  to  obtain 
a  consistent  estimate  for  the  AR  parameters.  In  Section  IV,  a  spectral  estimate  is  proposed 
for  estimating  the  power  spectrum  of  the  AR  series  when  the  series  is  observed  in  additive 
noise.  The  proposed  spectral  estimate  is  shown  to  be  a  consistent  estimate. 


®  Akaike,  H.,  “Power  spectrum  estimation  through  autoregressive  model  fitting.”  Ann.  Inst.  Statist.  Math., 
vol  21, 407-419, 1969. 

9  Kromer,  R.,  “Asymptotic  properties  of  the  autoregressive  spectral  estimator”  Ph.D.  thesis,  Dept,  of 
Statistics,  Stanford  Univ.,  Stanford,  CA,  1969. 

Berk,  K.N.,  “Consistent  autoregressive  spectral  estimates.”  Ann.  Statist.,  vol  2,  no.  3, 489-502, 1974. 

*  *  Kay,  S.M.,  "Noise  compensation  for  autoregressive  spectral  estimates,”  IEEE  Trans.  Acoust.,  Speech, 
Signal  Processing,  vol  ASSP-28,  no.  3, 1980. 

^Gingras,  D.F.,  “Asymptotic  normality  of  autoregressive  parameter  estimates  for  mixed  time  series.” 

Naval  Ocean  Systems  Center,  Tech.  Report  733, 1981. 


II  THE  AUTOREGRESSIVE  AND  AUTOREGRESSIVE-MOVING 
AVERAGE  PROCESSES 


Assume  we  have  observations 


ML- 


a  stationary  scalar  zero  mean  discrete 


parameter  time  series  generated  by  a  finite  order  autoregressive  (AR)  process 

Xt  -  alxt-l  "  a2Xt-2 - aMXt-M  =  et  0) 

where  the  innovations  sequence  of  random  variables  |  et  J  is  assume£* to  be  independent 
normal  identically  distributed  with  zero  mean  and  variance  op,  i.e.,  i.i.d.  N(0,  op-).  To 

guarantee  that  the  process  |  Xt  J  is  stationary,  the  roots  of  the  polynomial 
l-a,Z - aMZM 

must  lie  outside  the  unit  circle  on  the  Z-plane. 

The  autocovariance  function  can  be  obtained  by  multiplying  ( 1 )  by  Xt_k  and  taking 
expectations  term  by  term,  we  get 

rx(k)-aj  rx(k-l) - aM  rx(k-M)  =  0  k>0  (2) 

where  rx(k)  *  E  j  Xt_k  Xt  J  .  Note  that  the  term  E  |  Xt_^  et  J  does  not  exist  for  k  >  0. 
This  is  the  case  since  Xt_^  can  only  be  correlated  with  inputs  es  for  s  <  t  -  k.  If  we  evaluate 
the  autocovariance  function  (2)  for  k  =  1 ,  2,  —  ,  M  we  obtain  the  Yule-Walker  (Y-W)  equa¬ 
tions.  The  Y-W  equations  are  a  set  of  linear  equations  for  the  AR  parameters  in  terms  of  the 
autocovariances  rx.  In  matrix  form  we  have 


rx(°)  rx(l) 

rX  H)  rx  (°) 


rx(M-l) 
rx  (M-2) 


rX(D 

rX(2) 


rx(l-M)  rx(2-M) 


rx(0) 


rx(M) 


where  the  M  X  M  covariance  matrix  is  a  symmetric  real  Toeplitz  matrix.  When  sample  esti¬ 
mates  for  the  covariances  are  substituted  into  the  Y-W  equations,  the  solution  provides  con¬ 
sistent  estimates  for  the  AR  parameters.  We  can  write  the  Y-W  equations  in  terms  of  the 
estimates  as 


iH 

2.  VxlHO  -  ?x<k> 

j-l  : 


k  =  1,  2,  -  ,M 
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j  N-lkl 

where  ?x(k)  =  -  ^  xt  *t+|k|  . 

t=l 

Mann  and  Wald  1 13] ,  in  a  paper  dealing  with  linear  stochastic  difference  equations 
developed  least  squares  estimates  for  the  AR  parameters  and  proved  that  the  estimates  were 
consistent  and  asymptotically  unbiased.  They  proved  that  the  parameter  estimation  errors 
were  asymptotically  normal  and  calculated  the  asymptotic  error  covariance  matrix.  Mann 
and  Wald  [13]  also  developed  an  estimator  for  the  innovations  process  variance,  oe2. 


„  M 

a€ 2  =  ?x(0)  -  J  ?XG)  (5) 

j=l 


and  showed  it  to  be  a  consistent  estimate. 

The  power  spectral  density  for  an  AR  process  of  finite  order  is  written  as 


0X  (X) 


2ir  A(eiX)  A(e“iX 


) 


(6) 


M 

where  A(eiX)  =  1  -  ^  aj  exp  (ijX).  An  estimator  for  the  power  spectrum  of  an  AR  time 

j-1 

Series  that  is  often  used  is  (6),  with  estimates  for  o€2  and  the  AR  parameters  used  in  place 
of  the  true  quantities.  Akaike  [8] ,  Kromer  [9] ,  and  Berk  [10]  all  have  studied  the  asymp¬ 
totic  properties  of  this  spectral  estimator;  their  results  are  discussed  in  Section  IV. 

In  Section  III  it  is  shown  that  the  sum  of  an  AR  process  of  order  (M)  and  noise  can 
be  modeled  as  an  ARMA  process  of  order  (M,  M).  This  process  can  be  represented  by 


Yfal  Yt-1 - aM  Yt-M  "  ^t  “bl  ^t-l - bM  ’It-M 


(7) 


where  the  sequence  |  r?t  J  is  assumed  to  be  independent  normal  identically  distributed  with 

zero  mean  and  variance  o^2,  i.e.,  i.i.d.  N(0,  o^2). 

We  can  evaluate  the  autocovariance  function  for  this  process  by  multiplying  through 
by  Yt_k,  and  taking  expectations  term  by  term,  we  get 


E  (Y.-k  V, 
=  E 


}-alE(Yt-k  Yt-1  } - aM  E  {  Yt-k  Yt-M  } 

( Yt-k ^t )  -  bl  E  {Yt-k  ^t-1  J - bM  E  ( Yt-k  ^t-M  |  • 


(8) 
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Mann,  H.B.  and  Wald,  A.,  “On  the  statistical  treatment  of  linear  stochastic  difference  equations.” 
Econometrics,  vol  II,  173-200,  -<H3. 


Since  Yt_k  depends  only  on  inputs  t?t_j  for  t-j  <  t-k  it  follows  that 


1  I  (o  k>j 

E  Yt.kr,H  )  = 

(  rYtj(j-k)  k<j  . 

In  (8)  we  see  that  the  range  of  j  is  0  to  M,  thus  if  we  define  rY(k)  =  E  |  Yt_k  Yt  J  then  we 
can  write 

rY(k)  -aj  ry(k-l) - a^j  ry(k-M)  =  0  k  =  M  +  1,-,2M  (9) 

or  in  matrix  form,  we  have 


'ry(M)  ry(M-l) 

ry(lf 

’al" 

ry(M+l)" 

rY(M+l)rY(M) 

ry(2) 

a2 

ry(M+2) 

rY(2M-l)rY(2M-2)  - 

ry(M) 

-aM_ 

ry(2M) 

These  relationships  are  referred  to  as  the  “higher  order”  Yule-Walker  equations.  When 
sample  estimates  of  the  covariances  are  substituted  into  these  Y-W  equations  the  solution 
provides  estimates  for  the  AR  parameters  for  a  mixed  autoregressive-moving  average  time 
series. 

Lemma  1: 

Given  |  Yt  j  ,  observations  of  a  finite  order  ARMA  process,  then  estimates  of  the  AR 

(  iM 

parameters  {  a:  J  ,  estimated  using  the  “higher  order”  Y-W  equations,  are  consistent 
l  Jlj=l 

estimates. 

Proof: 

This  result  was  proven  by  Gersch  [  14] . 

The  problem  of  estimating  the  autoregressive  parameters  for  a  mixed  autoregressive 
moving-average  time  series,  when  the  process  order  is  known,  was  first  examined  by  Gersch 
[14].  Gersch  [14]  showed  that  when  the  “higher  order”  Y-W  equations  are  used  to  esti¬ 
mate  the  AR  parameters  the  estimates  are  asymptotically  unbiased.  The  structure  of  the 
asymptotic  error  covariance  matrix  was  also  evaluated.  More  recently,  Gingras  [12]  proved 
that  when  the  “higher  order”  Y-W  equations  are  used  the  AR  parameter  estimates  are 
asymptotically  jointly  multivariate  normal  with  zero  mean  and  finite  covariance  matrix. 


1  ^  Gersch,  W.,  “Estimation  of  the  autoregressive  parameters  of  a  mixed  autoregressive  moving  average  time 
series.”  IEEE  Trans.  Automat.  Contr.,  vol  AC-15,  583-588, 1970. 


The  asymptotic  error  covariance  matrix  was  evaluated  and  was  shown  to  be  equivalent  to 
the  result  of  Gersch  [  14] . 

For  the  mixed  autoregressive-moving  average  case  various  methods  exist  for  estimat¬ 
ing  the  moving  average  (MA)  parameters.  For  example,  Box  and  Jenkins  [15]  provide  a 
method  for  iteratively  calculating  estimates  for  the  MA  parameters  from  the  AR  parameter 
estimates  and  the  sample  covariance  estimates.  Similiarly,  Anderson  [16]  and  Hannan  [  17] , 
[18]  also  provide  methods  for  estimating  the  MA  parameters  of  a  mixed  time  series. 

The  power  spectral  density  for  an  ARMA  process  is  given  by 

,  ,  °n2  B(eiX)  B(e'iX) 

vyW  — z~  - rr - 

2n  A(elX)  A(e"lX) 


M 

where  A(e*x)  is  as  previously  defined  and  B(elX)  =  1  -  ^  bj  exp  (ijX).  TheMApara- 

j=l 

meters  jbj,b2,  -  ,  bj^  J  determine  the  zero  locations  and  the  AR  parameter  j  aj^,-,  aj^ 

determine  the  pole  locations.  The  composite  response  of  the  poles  and  zeros  determine  the 
spectral  density  for  the  ARMA  process. 


*  ^  Box,  G.E.P.  and  Jenkins,  G.M.,  Time  Series  Analysis:  Forecasting  and  Control,  San  Francisco:  Holden- 
Day,  1976. 

^Anderson,  T.W.,  The  Statistical  Analysis  of  Time  Series,  New  York:  John  Wiley  and  Sons,  Inc.,  1971. 

^  Hannan,  E.J.,  “The  estimation  of  mixed  moving  average  autoregressive  systems.”  Biometrika,  vol  56, 
579-593, 1969. 

^Hannan,  E.J Multiple  Time  Series,  New  York:  John  Wiley  and  Sons,  Inc,  1970. 
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III.  THE  AUTOREGRESSIVE  PROCESS  IN  NOISE  (AR  +  N) 


Assume  that  the  observed  discrete  parameter  time  series  is  generated  by  the  sum  of  a 
stationary  autoregressive  process  |  Xt  J  of  known  order,  M,  given  by 

M 

X,-  2  fj  XH  -  <r,  (ID 

j=l 

and  a  noise  sequence  |  nt  J  •  As  before,  assume  that  the  innovations  sequence,  jet  |  is  i.i.d. 
N(0,  of“),  that  the  noise  sequence  |ntJ  is  i.i.d.  N(0,  on^)  and  that  the  series  |  Xt  j  and  |nt  J 
are  independent.  Define  the  observed  series  |  Yt  j  by 

Yt  =  Xt  +  nt  02) 

where  |  Xt  |  and  |  nt  J  are  as  defined  above.  Also  define  the  series  |  vt  |  by 
M 

vt  =  Yf  |  aj  Yt-j  •  <13> 

j=l 

The  following  two  Lemma’s  are  results  developed  by  Pagano  [5] . 

Lemma  2: 

There  exists  a  sequence  of  random  variables  |  r/t  J  that  are  i.i.d.  N(0,  defined 
on  the  same  sample  space  as  J  Vt  |  and  constants  J  bj  | ._  j  such  that 

M 

vt  "  r?t  -  Mt-j  (14^ 

j=l 


where  {  v.  }  is  a  moving  average  series  of  order  M. 


5 

I 

'l 

. 


Proof: 

From  ( 1 2)  and  ( 13)  we  have 
M 

vt  =  -  ^  aj  (XH  +  nH)  a0  =  -1 
j=0 

M 

=  et  -  ^  aj  nt_j  .  (15) 

1=0 

Since  |  vt  J  is  stationary,  we  define 
rv(u)  =  E  jvtvt+uJ 

where  from  (13)  we  see  that  rv(u)  is  zero  for  |u|  >  M.  Therefore  |  vt  j  is  a  moving  average 
series  of  order  less  than  or  equal  to  M.  From  (14)  and  (1 5)  we  have 

rv(M)  =  °Tj2bM  =  °n2  aM  • 

Since,  by  definition,  aM  is  non-zero,  then  the  parameter  must  also  be  non-zero.  Thus 
|vt  |  must  be  a  moving  average  series  of  order  M. 

Lemma  3: 

The  series  |  Yt  J ,  as  defined  above,  is  a  mixed  autoregressive  moving  average  series 
of  order  (M,  M). 

Proof: 

Equating  (13)  and  (14),  taking  Z  transforms,  we  have 
Y(Z)  A(Z)  =  tj(Z)  B(Z). 

From  this  relationship,  with  Z  =  e*\  it  follows  directly  that  the  power  spectral  density  of 
the  process  { Yt  J  is  given  by 
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06) 


* 


*Ym  -  -*■ 


B(eiX)  B(e~iX) 
A(eiX)  A(e"iX) 


We  can  also  express  the  power  spectral  density  of  Yt 


as 


0y(A)  =  -T- 


2ir  2ir  A(eiX)  A(e"iX) 


(17) 


Thus  we  can  write 

a2  B(Z)  m~X)  =  o2  +  on2A(Z)A(Z*1)  . 

Therefore  if  ZQ  is  a  root  of  A(Z)  then  neither  ZQ  nor  Z0“*  is  a  root  of  B(Z).  Thus,  0y(x) is 
the  modulus  squared  of  the  ratio  of  two  Mth  degree  polynomials  with  no  common  roots, 
therefore  |  Yt  J  is  a  mixed  autoregressive-moving  average  series  of  order  (M,  M). 

As  a  result  of  Lemma  3  we  can  express  the  series  |  Yt  |  as 

Yt  "  al  Yt-1 - aM  Yt-M  *  **t-bl  ^t-l  - bM  ^t-M  • 

Note  that  since  the  AR  +  N  series  can  be  expressed  as  an  ARMA  series  the  “higher  order” 
Y-W  equations,  discussed  in  the  previous  section,  can  be  used  to  estimate  the  AR  parameters 
for  this  case. 

Equation  (17)  expresses  the  power  spectrum  for  the  |  Yt  J  process  in  terms  of  its 

components,  i.e.,  the  spectrum  for  the  noise  and  the  spectrum  for  the  AR  process.  We  can 
rewrite  (17)  as 


a  2  +  a  2  A(e‘X)  A(e_‘x) 

0Y(X>  =  - - - - - 

2ir  A(elX)  A(e"lX) 


(18) 


This  form  of  the  spectral  density  provides  insight  into  the  characteristics  of  the  AR  +  N 
process.  We  see  from  (18)  that  the  spectral  density  includes  both  poles  and  zeros.  We  also 
note  that  the  zeros  have  been  introduced  by  the  presence  of  the  additive  noise  and  that 
the  zero  locations  are  related  to  the  pole  locations  and  variances  o^2,  an 2.  In  Kay  [6]  the 
roots  of 


a2  +  on2  A(Z)A(Z_1) 


13 


are  examined  as  a  function  of  signal-to-noise  ratio.  It  was  shown  that  for  a  high  signal-to- 
noise  ratio  case  (i.e.,  oe^>  on^)  the  zeros  are  located  near  the  origin  on  the  Z*plane.  As 
the  signai-to-noise  ratio  decreases,  the  zeros  move  toward  the  pole  locations,  cancelling  the 
effect  of  the  poles,  producing  a  flat  noise-like  spectral  estimate. 
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IV.  A  CONSISTENT  AR  SPECTRAL  ESTIMATE 


When  the  observed  time  series  |  Xj  j  can  be  fitted  to  a  finite  order  AR  model  the 
following  AR  spectral  estimator  is  often  used 


♦x<*>  = 


a  2 


2ir  A(eiX)  A(e"iX) 


where  A(eiX)  =  1  -  ^  ak  exp  (ikX)  and  the  set  j  are  the  estimated  AR  para- 
k=l  k_1 

meters.  The  estimate  for  ae 2,  as  defined  by  (5),  is  a  function  of  the  estimated  AR  para¬ 
meters  and  the  sample  covariances.  Akaike  [8] ,  Kromer  [9] ,  and  Berk  [10]  have  studied 
the  asymptotic  statistical  properties  of  this  spectral  estimate  when  the  AR  parameters  are 
estimated  via  the  Y-W  equations.  Akaike  [8] ,  under  the  assumption  that  the  process  order 
M  is  finite  and  known,  proved  that  the  AR  spectral  estimate  is  asymptotically  normal  and 
evaluated  the  asymptotic  covariance  matrix  for  the  limit  distribution  as  N  -*■  <».  Kromer  [9], 
under  assumptions  relative  to  the  smoothness  and  boundedness  of  the  spectral  density, 
proved  that  the  AR  spectral  estimates  are  consistent  and  asymptotically  normal  as  first 
N  -*■  «*  then  M  -*• <».  Kromer  [9]  also  evaluated  the  covariance  matrix  of  the  limit  distribu¬ 
tion  and  provided  an  indication  of  the  rate  of  decrease  for  the  bias.  Berk  [  10] ,  under 
assumptions  similar  to  Kromer’ s  and  with  assumptions  on  the  asymptotic  rates  for  N  and  M 
proved  that  the  AR  spectral  estimates  are  consistent,  asymptotically  normal  and  uncorre¬ 
lated  at  different  frequencies.  Berk  [10]  and  Kromer  [9]  both  proved  that  the  asymptotic 
variance  for  the  spectral  estimate  is  the  same  as  that  for  periodogram  estimate  based  on  a 
rectangular  window  truncated  at  ±M. 

When  the  observed  time  series  I  Yt  j  has  been  generated  by  an  AR  process  plus  white 
noise,  we  showed  in  Section  III  that  this'  series  can  be  modeled  as  an  ARMA  process  with 
power  spectral  density  given  by 


B(eiX)  B(e~iX) 
A(eiX)  A(e~iX) 


(19) 


Methods  are  available  for  estimating  the  AR  and  MA  parameters  for  an  ARMA  series.  Asymp¬ 
totically  efficient  estimates  have  been  developed,  usually  through  the  application  of  iterative 
solutions  to  nonlinear  schemes  (see  Parzen  [  19] ,  Hannan  [  1 7] ,  [  18] ,  and  Anderson  [  16) ). 
Recently,  Hannan  [20]  developed  a  recursive  method  for  estimating  the  parameters  for  an 


19  Parzen,  E.,  “Efficient  estimation  of  stationary  time  series  mixed  schemes,’’  Stanford  Univ.  Tech.  Report 
#  1 6  on  Contract  Nonr-225(80),  1971. 

20  Hannan,  E.J.,  “Recursive  estimation  based  on  ARMA  models,”  Ann  Statist.,  vol  8,  no.  4, 762-777, 
1980. 


ARMA  series.  With  estimates  for  a^,  B(e*^)  and  A(e^),  one  could  construct  an  estimate 
for^y(A)  using  (19).  The  methods  available  for  estimating  the  MA  parameters  are  quite 
complex,  and  we  note  that  the  spectral  estimate  desired  is  that  for  not  <t> y(A). 

Therefore,  we  see  that  if  we  could  estimate  ae 2  and  A(e^)  from  observations  of  the  noise- 

corrupted  series  J  Yt  I  then  we  would  have  an  estimate  for  the  AR  spectrum  0^(X). 


ak 


)M 

1  k=l 


Pagano  (51  addressed  the  problem  of  estimating  the  parameters  on  ,  oe2  and 


for  an  AR  +  N  series.  Under  assumptions  that  the  noise  and  the  innovations 


series  are  normally  distributed  through  the  application  of  nonlinear  regression  methods  he 
proved  that  efficient  estimates  of  the  parameters  can  be  constructed.  Pagano  [5]  did  not 
address  the  estimation  of  the  spectral  density. 


The  objective  of  the  work  herein  was  the  development  of  a  consistent  spectral  esti¬ 
mate  for  the  AR  time  series  when  the  observations  are  corrupted  by  additive  white  noise. 
It  was  previously  shown,  in  Lemma  1 ,  that  consistent  estimates  for  the  AR  parameters  can 
be  constructed  through  the  use  of  the  “higher  order”  Y-W  equations.  Thus,  it  remains  to 
develop  a  consistent  estimate  for  the  variance  of  the  innovations  series,  oe 2,  from  observa¬ 
tions  of  the  noise-corrupted  series. 

It  was  shown  in  Section  III  that  the  power  spectral  density  for  the  AR  +  N  process 
can  be  written  as 


0y(A)  = 


+ - 

2ir  A(e 


iX)  A(e“iX) 


G 


We  now  use  the  linear  relation  of  (20)  in  conjunction  with  the  AR  parameter  estimates 
1a  )M 

{ ak  J  and  a  smoothed  periodogram  estimate  for  0y(  A)  to  develop  an  estimate  for  the 
*  •  k=l 

A 

variance  of  unobservable  innovations  series,  et.  Let  J^(X)  be  the  smoothed  periodogram 
estimate  for  Yt,  we  have 


N-l 


JN(X)  =  —  £  h(k  Bn)  fy(k)  exp  (-ikX) 

kMN-1) 


0 


where  fy(k)  is  the  covariance  estimate  given  by 


and  h(k  B^)  is  the  covariance  averaging  kernel  defined  as  the  Fourier  transform  of  the  weight 
function  defined  by  Brillinger  [21  ] .  The  sequence  B^,  N  =  1 ,  2,  •••  is  a  sequence  of  scale 
parameters  with  the  properties  B^  >  0,  Bjq  -*■  0  and  B^  N  -*•  «>  as  N  -*•  <». 


From  (20),  for  all  X:  =  -  ,  j  =  0,  1 ,  ••• ,  M-l ,  we  can  write  the  linear  relation 

M-l 


"'W  =  °"2  +  7^0 

A(e  J)  A(e  ■*) 


where  Uj  is  an  undefined  random  error  term.  We  propose  as  estimators  for  on^  and  the 
least  squares  estimates  derived  from  (22).  Define  $  to  be  a  (M  X  1)  vector  of  estimates 
2jt  Jn(Xj)  j  =  0,  1,  -  ,  M-l,  and  let  g(Xj)  =  A(eiXj)  A(e*iXj).  Define  the  (M  X  2)  matrix  H  by 


Ht  = 


1  1 


1  1 


•••  1 


8(x0)  sap 


g<XM-l> 


Define  0^  ^  [an^’°e^|  an<*  =  [  u0*  ul,‘'' ’  uM-l]  then  we  can  write  the  matrix 
equivalent  of  (22)  as 

<t>  =  H  0  +  u  . 

The  well  known  least  squares  estimate  for  0  is  then  given  by  the  solution  of  the  normal 
equation 

TA  t 

H)  0  =  Ht  f  . 

Define  7^  by 


then  the  solution  of  the  normal  equation  yields  the  following  relationships 


^  Brillinger,  D.R.,  Time  Series  Data  Analysis  and  Theory,  New  York:  Holt,  Rinehart  and  Winston,  Inc.,  1975. 


It  follows  that  the  least  squares  estimate  of  the  innovations  series  variance  is  given  by 


2»m  y‘  m  -2,fyyy'— 

k  *¥  ft  k  “V 

■  m  •  (i 


A  ? 

°e  = 


Lemma  4: 


1  1M 

Given  consistent  estimates  of  the  AR  parameters  |  ak  |  then  the  least  squares 


estimate  for  a  ^  as  given  by  (23),  is  a  consistent  estimate. 


Proof: 


We  have  from  Lemma  1  that  plim  &k  =  for  k  =  1,  2,  ••• ,  M,  thus  it  follows  that 

N  -*■  oo 


A(e,Xj)  A(e  ,Xj) 

N 

for  j  *  0,  1,  • ,  M-l,  and  correspondingly 
A(e'iXj)  — A(e'iXj)  . 

N-»  oo 

Therefore,  we  can  conclude  that 

2(v  «<xi» 

for  j  =  0,  1,  ,  M-l.  Given  the  smoothed  periodogram  estimate,  as  previously  defined,  we 

know  from  Brillinger  (21  ]  that  Ijj(X)  converges  in  mean  square  to  uniform  in  X, 

thus 

^n(X)  *Y*X> 
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uniform  in  X.  From  (23)  we  can  write 


plim  a  2  = 

N-»oo 

M 


«Xj)  2.K*  5T'  1 

mS  ^ 

j=o _ [ _ rO _ j=o 

1_  /] _ \2  j\_  1 _ \2 

M  Z  U(Xj)J  ”(m  Z  g(Xj)) 

j=o  '  J/  \  j=0  7 


(24) 


Using  the  relationship 

2ir  4>y(\)  =  on2  +  oe2/g(X) 


the  numerator  of  (24)  becomes 


sfeH  slop 

•  Ifer, '  •>  (sv)= '  •  -•»’  (I  k)' 


=  *2 


5  lfe)2-(sls^)2j  • 


(25) 


Substituting  (25)  for  the  numerator  of  (24)  we  have 


A  ^  ^ 

plim  o  4  =  o  4 


N- 

Lemma  5: 


M 


Given  consistent  estimates  for  the  AR  parameters  |  a^  j  and  the  consistent  esti¬ 
mate  for  oc2  then  the  spectral  estimate  for  the  unobservable  series  j  Xt  |  given  by 


*X<*>  ~ 


a  2 
ae 


2ir  A(e*^)  A(e“'^) 


is  a  consistent  estimate. 

Proof: 

The  result  follows  directly  from  the  consistency  of  the  estimated  AR  parameters  and 
the  estimate  for  oe 2. 
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